Combination of activation functions in extreme learning machines for multivariate calibration

a r t i c l e i n f o The key point in multivariate calibration is to build an accurate regression relationship between the predictors and responses. In this paper, we first use extreme learning machine (ELM) to build spectroscopy regression model. Then, we propose a combinational ELM (CELM) method in which the decision function is represented as a sum of a linear hidden-node output function (activation function) and a nonlinear hidden-node output function. As the output functions map the input spectral signal to linear and nonlinear feature spaces respectively, the proposed method can effectively describe the linear and nonlinear relations existed in spectroscopy regression by the CELM output weights vector which can be simply resolved by ridge least squares or alternative iterative regularization. The proposed method is compared, in terms of RMSEP, to PLS and ELM on simulated and real NIR data sets. Experimental results demonstrate the efficacy and effectiveness of the proposed method. According to Beer–Lambert law, the absorption of light in a medium is proportional to the pathlength and the concentration of the absorbing agent. That is, there is a linear relationship between absorbance and concentration when the pathlength keeps constant, which motivates the development of linear multivariate calibration (MVC) techniques [1], such as multiple linear regression (MLR), principal components regression (PCR) and partial least squares regression (PLS). However, the linearity of the Beer–Lambert law is limited by chemical and instrumental factors, such as, deviations in absorptivity coefficients at high concentrations, non-symmetrical chemical equilibrium, intermolecular reactions, existence of humidity inducing hydrogen bonding, changes in temperature, non-monochromatic radiation, scattering of light, fluorescence or phosphorescence of the sample, stray light, nonlinear detector response [2], etc. When the system exhibits strong nonlinear behaviors, classical linear methods may not completely present the relationship between the spectra and corresponding concentrations and thus would produce large errors. Many nonlinear techniques are developed, such as, artificial neural network (ANN) [2–4], support vector machine (SVM) [5,6], locally weighted regression (LWR) [7], nonlinear partial least squares (quadratic PLS [8], spline PLS [9], kernel PLS [10]), etc. These methods may perform well on nonlinear data but are computationally more complex than linear methods and have the limitation of being prone to overfitting. The search of hyperparameters in SVM is time-consuming, and model parameters in LWR are usually less stable. Quadratic PLS can only model weak nonlinearities and is lack of flexibility in modeling the complex …